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The objectives of this study are twofold: a) 
measuring students • expressive engagement levels while increasing the 
teachers* use of two teaching strategies. Personalizing the Task (PT) 
and Recognizing Individual Achievement (RIA) ; and b) testing the use 
of observation, feedback, and training as a practical model for 
changing teacher behavior. Expressive engagement was defined here as 
active student involvement in learning task. Trained observers 
recorded the use of PT and RIA by three third and fourth grade 
teachers and the levels of engagement for 30 low-income students who 
were diagnosed as potential academic and discipline problems* During 
the 5-day experimental period for each strategy, observers and 
teachers met to discuss strategy frequency and suggest ways of using 
selected strategies. Observer feedback was effective in modifying 
teacher behavior. The frequency of both strategies increased while 
student engagement levels showed no consistent relationship to 
strategy use. Strong individual differences in the implementation of 
these strategies was proposed as a possible explanation. (Graphs and 
charts of statistical data are presented in the text. The appendixes 
include definitions and explanations of teacher strategies, student 
behavior, and interobserver agreement tables.) (Author/BRB) 
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INTRODUCTORY STATEMENT 



The Center's mission is t.o improve teaching in American schools. 
Too many teachers still employ a didactic style aimed at filling passive 
students with facts. The teacher's environment often prevents him from 
changing his style, and may indeed drive him out of the profession. 
And the children of the poor typically suffer from the worst teaching. 

The Center uses the resources of the behavioral sciences in pur- 
suing its objectives. Drawing primarily upon psychology and sociology, 
but also upon other behavioral science disciplines, the Center has formu- 
lated programs of research, development, demonstration, and dissemination 
in three areas. Program 1, Teaching Effectiveness, is now developing a 
Model Teacher Training System that can be used to train both beginning 
and experienced teachers in effective teaching skills. Program 2, The 
Environment for Teaching, is developing models of school organization 
and ways of evaluating teachers that will encourage teachers to become 
more professional and more committed. Program 3, Teaching Students from 
Low- Income Areas, is developing materials and procedures for motivating 
both students and teachers in low-income schools. 

The study summarized here represents part of the work of Program 3 
on the effect of various teaching strategies on student engagement. 
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ABSTRACT 



The objectives of this study were to measure students' expressive 
engagement levels while Increasing their teachers* use of 2 teaching 
strategies » and to test whether observation, feedback, and training, 
used together, are a practical model for promoting changes In teacher 
behavior. Expressive engagement was defined as active student attention 
to or Involvement In a learning task. The strategies were Personalizing 
the Task and Recognizing Individual Achievement. The design followed an 
N s 1 format. 

Trained observers recorded the use of the 2 strategies by 3 third- 
and fourth<-grade teachers and the associated levels of engagement for a 
sample of 30 students, 10 In each of 3 classrooms. In low-lncome area 
schools. The teachers picked pupils they had diagnosed as potential 
academic or discipline problems. Baseline data were gathered for eight 
days (16 hours) of observation time In each classroom. During the five- 
day (10 hour) experimental period for each strategy, observers met dally 
with each teacher to report on strategy frequency and engagement of stu- 
dents and to suggest ways to increase the teacher's use of the selected 
strategy. 

Observer feedback was effective In modifying teacher behavior. The 
frequency of the Personalizing the Task strategy showed an overall average 
Increase of 51.8 occurrences per two-hour observation period; the fre- 
quency of Recognizing Individual Achievement showed an overall average 
Increase of 10.3 occurrences per observation period. However, the data 
on student engagement showed no consistent relationship to the Increases 
in strategy use. 

There was evidence of strong individual differences in the way strat- 
egles were Implemented, which may account for a lack of consistency in 
engagement level changes. 

The study demonstrated to the teachers that they could modify their 
instructional techniques by objectively monitoring student responses in 
their classrooms. The method gave them an objective view of their in- 
fluence on their classes. All the teachers reported that they enjoyed 
the project. They seemed to be motivated and stimulated by ideas from 
the observer/teacher interaction. 
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CHAPTER I 



THE PROBLEM 

• Introduction 

Studies in education have born directed toward changing teacher 
behavior. Extensive work has not been done in a systematic way to 
examine what effects teacher changes have on pupil behavior* Thus, 
there is a real need to study teacher behavior and student responses 
in the same context. In order to introduce effective changes in ex- 
pressive classroom engagement (active student attention and involve- 
ment), teachers need to know that changes in their behavior can have 
positive effects on student learning and motivation. They also need 
to know the probable consequences of specific changes so that they can 
select those most appropriate to their situation. This study examines 
the impact of changes in teacher behavior upon the level of expressive 
student (classroom) engagement while developing practical methods for 
behavior change and pupil feedback that are sensitive to the uniqueness 
of individual classroom interactions. 

Theoretical Rationale 

Reform in teacher education has concentrated on restructuring 
training programs (Smith, Cohen and Pearl, 1969; Ccwibs, 1968) or on 
developing teacher qualities which are assumed to be desirable and 
helpful in promoting student motivation and learning (Borg, Kelley, 



Langer and Gall, 1970; Berliner, 1969; Joyce, 1967). Most of these 
approaches evaluate results on the basis of criteria established by 
outside agents (supervisors, educational theories, etc.)* Many of them 
do not include pupil behavior as an inqportant outcome. 

Some attempts have been made to deal with pupil responses by 
studying the interaction between teachers and their students. Flanders 
C1964) developed Interaction Analysis to provide teachers with an 
analysis of the effects of their own teaching behaviors on pupils. 
These effects, however, are often based on assumptions that certain 
behaviors on the part of the teacher lead to positive interactions; 
seven of Flanders* ten categories measure teacher verbal behavior while 
only two deal with pupil talk. Davidson (1967) used a modified inter- 
action analysis system to study the effects on pupils* level of thinking 
when teachers were provided with objective feedback data from their 
instruction in reading. He found that pupils made more critical 
thinking responses, fewer literal responses, and fewer nonproductive 
responses when their teachers received feedback. 

A few studies have shown that students can strongly influence 
teacher behavior. Gage, Runkel and Chatterjee (1960) demonstrated that 
when teachers were given data based on pupil ratings of their teacher 
and an ideal teacher they changed their behavior to more closely ap- 
proximate pupil ratings of the ideal. Jenkins and Deno (1969) determined 
that student classroom behavior was a powerful influence on teachers* 
self evaluation. When student ••actors" showed interest and excitement 
their teachers found teaching enjoyable, predicted they would be 



effective teachers, and thought that their students learned more thar 
the teachers whose students acted uninterested and unexcitec?. The 
authors suggest that teachers suspend judgments of their effectiveness 
until trained to assess student beha^ ^ ^ lively. A supporting 
study by Klein (1971), whose experimental conditions were similar to 
those of Jenkins and Deno, also showed that student behavior influenced 
the verbal and nonverbal behaviors of teachers. Criteria used for 
evaluating teacher changes were taken with an interaction analysis 
Instrument from classroom observations of teacher behavior. 

These findings suggest that teacher and pupil behavior affect 
each other. The question of whether or how much student engagement 
can be positively affected by manipulating the occurrence of specific 
teacher strategies remains unanswered. It is the purpose of this 
study to shed some light on this question. 

Definition of the Research Problem 

Hess (1973) studied low-income classroom settings to identify 
and describe strategies teachers use which are associated with high 
levels of student attention and involvanent (engagement). Teachers 
and pupils were observed simultaneously and behavioral indicators of 
student engagement were matched with teacher use of twenty-three verbal 
and nonverbal strategies. An assessment of the relative effectiveness 
of each strategy was made. 

The Hess data provide the background for this study by de- 
scribing strategies and engagement found in low- income classrooms/ The 
logical extension of this is a) cu manipulate teacher use of various 

^The study referred to as Hess (1973) summarizes the resul^s of 
work done in 1971-72. 



strategies to determine their relationship to student engagement; 
and b) to discover ways the data from the larger study have meaning 
for individual teachers on a practical, applied level. 

Sidman (1960) supports the rationale for exploring the con- 
ditions under which a phenomenon occurs as a means of building a 
foundation of specifics upon which scientific investigation can rest. 
He suggests that such data are of consequence only insofar as they 
establish or refute the soundness of other data. Experiments are 
important not only to test hypotheses but to collect data for theo- 
retical formulations. The latter function is especially useful in 
clearing away the cobwebs and building firm foundations in the mass 
of current educational research. 

The fact that teacher behavior can be changed with techniques 
of behavior modification has been well documented (Baer, Wolf and 
Risley, 1968; Cooper, Thomas and Baer, 1970; Schutte and Hopkins, 1970; 
etc.). This study concentrated on the amount of change in the two 
chosen strategies and on what effect changes in strategy frequency had 
upon student engagement. 

The independent variables of the first aspect of the study 
were using feedback and reinforcment to change teacher behavior; the 
dependent variable was teacher use of two selected strategies. The 
second aspect employed teacher use of two selected strategies as the 
independent variable and expressive student engagement as the de*- 
pendent variable. 

The specific questions posed were: 



1. How much can selected strategies be altered with the use 
of feedback, reinforcement, and other established behavior modifica- 
tion techniques? 

2. Are there systematic changes in student responses as a 
result of changes in the teacher's behavior? 

3. What are the problems encountered as a result of applying 
data from a large group design to individuals? 

4. Is the method of this study feasible for in-service 
t-r^ining? 

Treatment Rationale 

Training teachers to use selected strategies involved three 
aspects: (1) setting goals, (2) planning increases in selected 
strategies in daily lessons, and (3) receiving daily feedback on 
positive or negative consequences of strategy use on student engage- 
ment. In this study student engagement is defined as student atten- 
tion and task involvement. 

Teachers were informed of all phases of the research and took 
an active part in all planning sessions. The concept of experi- 
mentally manipulating variables and examining outcomes was shared 
with them. The importance of this approach when dealing with train- 
ing methods has been eloquently expressed by Argyris (1968). He 
states that research in field settings should involve subjects 
directly. It should motivate them by being relevant to their needs 
and by providing them with profits beyond ordinary feedback of 
results. 
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Receiving daily feedback in relation to established goals is 
seen as critical to success in changing teacher behavior. Biel (1962) 
and MacPherson, Dees and Grindley (1948) demonstrated that the role 
of feedback was a prerequisite for improvement or change but worked 
successfully only when accompanied by clearly defined goals. Feedback 
alone does not reinforce learning. The learner must be aware of how 
much closer to the desired goal each of his attempts is. 

Other studies lend further support to the value of immediate 
feedback. Packard (1970) found that teachers skeptical of behavior 
modification techniques gave the experimenters full cooperation when 
they became aware of dramatic changes in student behavior. Hall, 
Lund, and Jackson (1968) informed teachers participating in the re- 
search of daily pupil progress by showing them charts of observed 
behaviors. In addition, weekly conferences were held in which pro- 
cedures were discussed and teachers praised for bringing about desired 
changes in pupil behavior. Although this procedure was incidental 
to the study, it seems to have been of central importance to its 
success. Three beginning teachers who were having trouble with class- 
room control were trained by Hall, Panyon, Robon, and Broden (1968) 
to reinforce classroom study behavior systematically. Data on the 
study behavior of pupils in each class were recorded by trained ob- 
servers. Part of the treatment consisted of the daily presentation 
of these data to the teacher. This approach seemed to be extremely 
effective in changing teacher behavior. 

Feedback to teachers in this study will be based on behavioral 
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observations of the students. This approach is supported by the work 
of Tuckman and Oliver (1968) who examined feedback as a function of 
source. One group of teachers received feedback only from students, 
a second group from supervisors only, a third group from both students 
and supervisors, and a fourth group, idiich served as a control, re- 
ceived no feedback at all. The results (measured by student ratings 
across a twelve week interval) showed that feedback from students led 
to positive changes in teachers. Supervisor feedback combined with 
student feedback added nothing, while supervisory feedback alone led 
to changes in a negative direction when compared to the control group. 
The experimenters concluded that teachers were defensive and even 
hostile to supervisory attempts to shape their behavior. The best 
source of feedback appears to be the students, although this is often 
overlooked in teacher training. This study used behavioral ly based 
student feedback as a basis for changing teacher behavior and for 
the maintenance of those changes that have positive effects. 

Rationale for the Experimental Design of the Study 

An nsl format with replications was chosen as the most ef- 
fective experimental design for meeting the objectives of this study. 
The work of Hess (1973) made available important generalities about 
teaching strategies and their association with different levels of 
pupil engagement. This study seeks to expand his work by exploring 
possible causal relationships between selected strategies and student 
engagement and by developing a widely applicable method for training 
teachers that is sensitive to individual strengths and weaknesses. A 
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comparative group design would be inappropriate because the results 
are often not applicable to individual patterns of behavior except 
to the extent that they confonn to hypothetical averages (Zifferblatt , 
1972). Since this is an attanpt to develop individualized training 
programs, normative comparisons are irrelevant. 

N^l designs are based on as rigorous research methods as their 
statistical counterparts. They both involve systematic data collection, 
observation, experimental control, and analysis and synthesis of data 
CFrey, 1972; Lackenmeyer, 1970; Holt, 1970). One major difference 
between the two approaches is that a comparative research design is 
relatively inflexible and must be carried out as originally planned. 
It doesn't account for unforeseen events. In contrast, an n^l design 
allows the experimenter to stop at any time to (1) examine interesting 
discoveries, (2) correct methodological flaws, and (3) to vary experi- 
mental conditions in his search for optimal results (Skinner, 1956). 

The main focus of the n^l design in this study was to show 
that observed changes in pupil behavior resulted from the treatment 
intervention (e.g. changes in teacher use of selected strategies) and 
not some uncontrolled variable. Several techniques were adapted and 
applied to individuals since the aim was to develop an effective pro«^ 
gram for changing behavior and assessing the outcome (Hanniim, 1972). 
Individual variance was studied carefully to determine its source and 
influence on treatment variables, not eliminated or averaged out as 
between subject variance in group studies (Browning and Stover, 1971). 

The study of individual variability can make an important 



contribution to general educational research, especially when applied 
to confusing or contradictory results of field studies. This kind 
of research, while seldom producing spectacular findings, functions 
to establish a reliable methodology for the pursuit of more general 
problems (Sidman, 1960). 

The results of an n=l study can not be generalized to a 
larger population, but this is not the objective here. The importance 
of this approach for general use is in the development of flexible 
training methods that are widely applicable while accounting for 
individual differences. 



CHAPTER II 



DESIGN OF THE STUDY 

Overview 

This study was an attempt to increase teacher use of selected 
strategies and to examine the impact of the increases on expressive 
student engagement levels in natural classroom settings. 

A base rate of teacher use of the strategies and of the level 
of student engagement was measured during the first eight days of ob<* 
servation (16 sessions). Following this, a trainer worked intensively 
with a teacher to increase the use of one strategy for five days (10 
sessions) . Classroom observers charted the frequency of strategy use 
and student engag^ent levels and gave the teacher daily feedback and 
training. This process was repeated for the second strategy. Replica- 
tions occurred with two other teachers following the same procedure 
and reversing the strategy order in one case as a check for possible 
additive effects. At the end of the 18 day (36 session) experimental 
period, teachers were given training in collecting behavioral data on 
student engagement levels. A two day (bix session) follow-up again 
monitored and charted the use of both strategies and student engagement 
levels 14 days after the experimental period to assess longer term 
results of the treatment intearvention.^ 

^The author* s original plan included three selected strategies 
to be counterbalanced by the three teachers, a 10 day baseline, and a 
three day follow->up. The revisions were dictated by economic constraints* 
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The Sample 

The teachers in the sample were selected from the group of 24 
3rd and 4th grade teachers who participated in the Hess study of 
teacher strategies and student engagement in low income areas during 
1971-1972. Three interested teachers whose strategy profiles for 
71^72 showed low occurrences of the selected strategies were chosen 
for the present sample. 

A sample of ten pupils was chosen from each classroom. Each 
teacher picked pupils they had diagnosed as potential academic or 
discipline problems. The same pupils were watched consecutively 
throughout the experiment so that individual patterns of expressive 
student engagement could be analyzed. 

I nstrumentation and Procedure for Collecting Data 

The instruments used in this study were designed specifically 
for the Hess project on teacher strategies and student engagement. 

The Teacher Strategies Observation Instrument lists 23 strat- 
egies derived from theory and research on human learning, motivation, 
and teaching. A list of all the strategies with definitions, examples 
and a copy of the instrument can be found in Appendix A. Two of the 
23 listed categories were studied. 

The Student Engagement Observation Instrument provides be- 
havior-based measures and global assessments of student engagement in 
the classroom. The. sex, ethnicity, direction (the nature of the task 
to which the pupil is attending) and grouping (small, large, or dyadic 
for each child is also recorded. A list of the behavioral indicators 
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with definitions, examples, and a copy of the instrument will be 
found in Appendix B* 

An engagement score was plotted across the sample of 10 pupils 
to provide an average measure of the level of engagement at any given 
time. This score could range from 0 to +2 and was used as a quick 
assessment of the effect of changes in teacher behavior in order to 
give the teacher immediate feedback. 

Trained observers collected the classroom data. Two observers 
attended each observation session: one monitored student activity, 
the other observed the teacher. The observers were informed of all 
aspects of the research and consulted the teacher daily with the Experi- 
menter in individually arranged training sessions to provide feedback 
and discuss strategy use for the next session. 

Description jof the Independent and Dependent Variable,? 

The two strategies of interest in this study were Personal- 
izing the Task (PT) and Recognizing Individual Achievement (RIA) . They 
functioned as dependent variables when changing teacher behavior with 
feedback and reinforcement was the independent variable, but became 
independent variables when expressive student engagement was used as 
the dependent variable. 

PT and RIA were selected for this study because of their 
potential for providing irdividual and relevant learning experiences 
for students. It is this experimenter's bias that such an approach 
may be particularly rewarding in low- income classrooms* The frequency 
of the occurrence of these strategies during 10 hours of observation 
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in the Hess study was quite low (Hess, 1973). One of the questions - 
posed for this study was whether increases in these strategies would 
be associated with increased expressive student engagement. Both 
strategies and engagement were defined as observable behaviors; thus 
frequencies could be accurately quantified. 

Personalizes the Task is a strategy which transposes academic 
concepts into comprehensible terms by relating them to the students* 
or the teachers* personal experience. An example would be teaching 
map reading by having students locate their own homes on a detailed 
city map. 

Recognizing Individual Achievement is an affective feedback 
Strategy which indicates to the student that he has performed well, 
made an unusual contribution, or has in some way accomplished more 
than the usual standards of accomplishment. For example, the teacher 
might tell a child he has done such a great job on his arithmetic 
and imderstands it so well that she would appreciate his helping a 
child who is having trouble. 

The strategies chosen are less tightly controlled than those 
normally manipulated in an intensive experimental design. They were 
picked in the context of the larger Hess design from which they were 
selected. It was decided to sacrifice the exacting precision of a 
narrowly defined variable in order to examine how this approach would 
work in a natural setting. 

The dependent variable for the second aspect of this study 
was the expressive engagement level of the student sample when a 
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selected strategy use was increased by the teacher. Expressive en- 
gagement is defined as active involvement (motor or verbal) by the 
student in the attention or involvement process. The time of day and 
the type of lessons observed were held constant to minimize competing 
factors in the experimental situation. Language Arts and Math were 
the lessons studied as they were the two subjects taught daily in 
which the whole class could easily be grouped together. 

The frequency of occurrence of both student engagement and 
teacher use of strategies were plotted and graphed daily to provide 
immediate knowledge of results for teachers and experimenter. 

Data Collection and Treatment Procedure 

A baseline of teacher strategies and student engagement was 
recorded during the first eight days (16 sessions) of observation. 
This extended period was necessary to calculate a stable base rate in 
the complexities of a natural setting further confounded by the pre- 
sence of observers (Browning and Stover, 1971). 

Following the establishment of base rate behaviors, each 
teacher was presented with the baseline data collected in her class- 
room. Daily goals were set for increasing a selected strategy and 
specific suggestions for implementation were discussed. The teacher 
received daily feedback both on her performance (rate of occurrence 
of strategy compared to base rate) and on concurrent levels of pupil 
engagement. This process was repeated for the second strategy. 
Replications of this design occurred with two other teachers for one 
of whom the sequences of strategies were interchanged as a check for 
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practice and additive effects. Reversal techniques were not used to 
establish causality because they present both ethical and practical 
problems in school settings. 

A two day (four session) follow-up observation period monitored 
and charted the use of the two strategies and expressive student en- 
gagement levels two weeks after the experimental sessions were 
completed. 

During the course of the experiment an anecdotal log was kept 
by the experimenter and some observers. Unanticipated factors, changes 
in procedure, teacher and observer reactions, and decision-making 
processes were recorded in the log. 

Inter-Observer Agreement 

An index of inter-observer agreement for this study was ob- 
tained from data provided by an additional observer (calibrator) in 
the classroom during four of the observation times. The calibrator 
rated behavior simultaneously with an observer for 10 minutes using 
the Student Engagement Instrument or the Teacher Strategy Instrument. 
Each observer was calibrated twice on each instrument, once during 
the collection of baseline data and again during the experimental 
period. 

Observer and calibrator agreements and disagreements on each 
instrument were tallied and Scott's (19S5) formula was applied to 
obtain a percentage of agreement figure. This procedure allows for a 
correction for chance agreement in the final figure. 

Inter-observer corrected percentage of agreement for all 
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strategies on the Teachc^r Strategy Instrument for the baseline data 
collection was .84 for all observers combined* The range was from 
•80 to •96* During the experimental period the mean was .88 and the 
range was from .84 to •94* 

Inter-Observer corrected percentage of agreement for all cate- 
gories on the Student Engagement Instrument for the baseline data 
collection was .82 for all observers combined. The range was from 
• 76 to ,90^ During the expc^rimental period the mean was ^90 and the 
range from .78 to .98^ 

Tables showing all the percentages for each strategy in the 
teacher instrument and each category in the student instrument are 
included in Appendix €• 

Analyses 

Since the focus of this study was on individual variability 
rather than statistical prediction^ the data were analyzed by simple 
graphic presentations • Figures 1-3 show teacher strategies and student 
engagement for each of the sample teachers* Figures 4-S illustrate 
the intensity level of the student engagement recorded. 

Each Oi^currence of teacher use of selected strategies was 
added per day and across all the days for a specific period^ then 
divided by the number of days in that period* For example^ Figure 1 
shows for Teacher 1 that in language arts she used FT an average of 
five times per period during the eight day baseline period and an 
average of 58 times per period during the five day PT experimental 
periods 
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The procedure for calculating student expressive engagement ' 
was 1) to add the number of occurrences of expressive engagement for 
each observation period, 2) to divide each total by the number of 
observations made in that session (e.g. ideally 90 but often less due 
to absence, etc.)* 3) to add the daily computations together for each 
period (eight for baseline, five for each experimental period, two 
for follow*up) and 4) to divide the total for each period by the 
number of days in that period for an average percent of expressive 
engagement . 

Percentages were computed because of possible absenteeism 
among the student sample. For example. Figure 1 shows that the stu* 
dent sample in the class of Teacher 1 during the baseline period (in 
Language Arts) was expressively engaged 42 percent of the time ob- 
served. During the PT experimental period they were expressively 
engaged 62 percent of the time. 

A second analysis of engagement is concerned with changes in 
the levels of intensity (see Figures 4 and S). Each observation of 
engagement was scored with a 'I'l or a -^2 to indicate the level of 
intensity of engagement (-t-1 standing for interest and involvement and 
^2 standing for intense interest or involvement). Figures 4 and 5 
show the daily percentage of '•-2 scores for the sample students 
(Figure 4 refers to Language Arts and Figure 5 to Math) . 
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CHAPTER III 



RESULTS AND DISCUSSION 

* 

Impact of Interventior on Selected Strategy Use 

The data show that both selected teacher strategies were in- 
creased with feedback and training. All three teachers increased their 
use of Personalizing the Task during the experimental periods in 
Language Arts and Math (see Figures 1 and 3) . The increases from 
baseline to experimental periods extended from 14.4 (with a range of 
8 to 48) to 73.9 (with a range of S3 to 90) mean occurrences per 
observation period. The three teachers also increased their use of 
Recognizing Individual Achievement during t experimental periods 
in Language Arts and Math (see Figures 1 and 3). The increases from 
baseline to experimental periods extended from 6.8 (with a range of 
5 to 11) to 13.8 (with a range of 8 to 19) mean occurrences per ob- 
servation period. Differences in subject matter did not seem to be 
a great influence on the use of the two strategies except that PT 
was greatest in Math during the experimental periods and was lower 
than Language Arts in the follow-up (Figures 1 and 3). 

It will be noted that PT changed much more dramatically than 
RIA. PT appears more amenable to change than RIA. All three teachers 
complained that they felt RIA was contrived if done oftener than 
occurred naturally for them. They strongly resisted using more RIA. 
Teacher 1 said she felt it went against her character to give praise 
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in public because she usually reserved it for private conferences* 
Observers also reported that increases in RIA seemed "out of character" 
for Teacher 2. 

This difference may be a function of the nature of the two 
strategies. It is possible that the changes are more equivalent than 
the graphs illustrate. PT is a more global strategy— it can be an 
entire lesson and once set in motion it can run continuously through- 
out the period without conscious teacher effort. RIA is more specific 
in that it involves communicating exclusively with individuals, is 
not part of the lesson plan, and is difficult to use frequently 
without sacrificing the class as a whole and without becoming re- 
dundant or phoney. 

This situation exemplifies one of the problems of implementing 
pure experimental designs in a natural setting. Intervening enough to 
equalize the strategy frequencies would have distorted the natural 
setting beyond recognition. The aim was to deal realistically with 
behavior that occurs spontaneously in a normal classroom. 

Relationship Between Strategy Changes and Engagement 

Global changes in engagement levels (Figures 1, 2 and 3) show 
the percentage of time the students were expressively engaged. These 
data show a slight upward trend in expressive engagement when selected 
strategies are increased. 

Changes in the intensity level of expressive engagement are 
shown in Figures 4 and 5 which chart the progression of expressive 
engagement rated at a 2 level of intensity. Again, the variability 
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is pronounced: for some teachers at certain times intensity 2 ratings 
increased during the experimental periods. There is some evidence 
that 2 ratings will stay consistently higher in different subject 
matters for different teachers. For example. Teachers 1 and 3 had 
more 2*s in Math than Teacher 2. Teacher 2 had more 2 ratings in 
Language Arts than in Math. 

The data do not show a clear relationship between strategy 
changes and expressive student engagement. So much variability 
exists that changes in expressive student engagement as a result of 
changes in teacher behavior seem to be different for each individual 
and in each situation (see Figures 1, 2 and 3). Teacher 1 had the 
most noticeable change in expressive student engagement as a result 
of her experimental behavior. Expressive engagement rose dramatically 
in her class. Teachers 2 and 3 had more variable results even though 
Teacher 3 showed the largest increase and retention of PT of all the 
teachers. It is clear that the increased frequency of strategy use 
by teachers in this study did not consistently affect the expressive 
engagement level of the students. 

Problems Encountered with an Experimental Design 
in a Natural Setting 

A number of problems encountered in applying experimental 

techniques to a natural setting involved teacher variables. All three 

teachers reported changes in mood due to weather conditions, day of 

week, time of day, and frustration with individuals. Observers noted 

distinct differences in the emotional tone and learning climate of 
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each class. All of these variables influenced the way teacher strat- 
egies were received by the students. Even though all teachers used 
the selected strategies very little during the baseline and all in- 
creased usage during the experimental periods, individual differences 
in style and expression appeared to produce differential results for 
each teacher. For example, it was reported by observers that one 
teacher repeatedly used RIA as a tool for subtle discipline of the class 
or as a command to get the class to the level of control she desired. 
She would say things like, "So and So is working so well and has already 
finished the first problem. Why can't the rest of you do as well?" 
Negative feedback from observers had no apparent effect on this be- 
havior. 

All the teachers increased their use of PT substantially; 
some relinquished their control over the class much more than others 
while doing so. Teacher 1, for example, established a climate in 
which the students were free to express themselves even though she had 
some reservations about the change. Teacher 3 used PT extensively 
but never really "let go" of her control of the situation. This 
anecdotal data suggests that factors other than the simple strategy 
frequency strongly influence student reactions and may account for some 
of the contradictions in engagement findings. 

A second group of problems were student related. The fact 
that a stable base level on expressive student engagement was not 
established and that variability was apparent throughout all the 
observation periods indicate that parameters other than teacher 
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strategies were at work. Obvious differences in achievement and skill 
levels may account for part of the variability here. 

Another set of problems relates to the difficulty of measuring 
expressive student engagement in a natural classroom setting. The 
need for sampling precludes recording the moment-to-moment engagement 
of an individual child. Student engagement can also be a function of 
variables such as interactions with other students, physical and emo- 
tional states, attention needs and home conditions. It is extremely 
difficult to isolate the variables affecting the student other than 
those directly executed by the teacher. This situation is further 
complicated by the necessity of sampling ten different students during 
an observation time. 

Another problem in gathering engagement data is that some 
children learn early to look and act as if they were engrossed in a 
task when in fact they are not. Thus the behavioral indicators on 
the Student Engagement Instrument may be misleading in some cases. 

In addition, individual differences in student responses may 
make objective quantification of intensity 2 ratings almost impos- 
sible. For example, cautious hand raising for a timid child may well 
be an indication of intense engagement, while energetic motion from 
a chronic arm-waver may express moderate or low engagement. In the 
observation procedure, however, these differences would not be re- 
corded accurately because the focus is on the behavior itself. 

Discussion 

The results of this study indicate that it is much easier to 
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increase teacher behavior with feedback and training than to affect 
expressive student engagement with strategy change. Obviously, a 
teacher has more control over her own behavior than that of her stu- 
dents and is therefore more susceptible to prediction and change. In 
addition, working with the teacher involves a one-to-one interaction 
not possible with a sample of pupils. Student behavior may be af- 
fected by many variables other than teacher behavior. Therefore, it 
is not possible to predict student behavior as a function of teacher 
behavior in a natural setting with behavioral indicators such as 
those used in this study. 

These conclusions are supported by the tremendous variability 
in student data. Table 1 shows individual patterns of engagement 
Onean E) , +2 intensity ratings (mean +2) , and engagement fluctuations 
(mean /) (number of times the engagement level changed during an ob- 
servation) for the student sample in Teacher 3*s class. Student 1, for 
example, in Language Arts during the RIA Experimental period had a 
mean engagement level of 1.1, a mean +2 intensity level of 1.6, and a 
mean fluctuation rate of 3.6. Figure 7 graphs the profiles of the 
pupil with the lowest number of fluctuations in engagement (student 
8) and the pupil with the highest number of fluctuations (student 9). 
In both cases, much fluctuation in engagement occurs. The variability 
is as much in evidence during the baseline as during the experimental 
periods, indicating that other parameters are at work. 

Expressive student engagement as it has been defined in this 
study, can be seen as the result of a complex set of variables, some 
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of which have been discussed earlier in this report. Isolating and 
measuring single variables, other than relatively simple teacher be- 
havior, is extremely difficult, but may be essential to an under- 
standing of the dynamics of natural settings (Sidman, I960). 

There is, however, a basic dilemma facing experimental workers 
in natural settings. Isolation and control of variables affecting 
both student and teacher behavior may well be essential for measure- 
ment, but the process of achieving such control can destroy the 
validity of the findings and make generalization practically impos- 
sible. In this study, for example, it would have been possible to 
train all three teachers to use the strategies under consideration in 
precisely the same way, thus eliminating one source of variation. 
However, this procedure would also place some limitations on the 
usefulness of the training technique with other teachers, since the 
objective was to develop a technique for change that could be helpful 
to a teacher yith a minimum of "outside" intervention. 

Another aim of this study was an in-depth examination of 
teacher strategies and student behavior in light of the findings of 
the larger Hess (1973) study which involved 23 strategies and 24 
classrooms. The lack of clear patterns of strategy- engagement rela- 
tionships reported by Hess may be a result of the individual differ- 
ences and resultant variability demonstrated for both students and 
teachers in this study. The anecdotal material on teacher strategy 
use clearly indicates unquantified differences among teachers. The 
intensive charting of engagement levels of students over time amply 
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demonstrates the existence of other variables at work in the classroom. 

In summary, it seems clear that in spite of the elaborate 
instrumentation of both the current study and the Hess (1973) project, 
teacher strategy use cannot be shown to have predictable effects on 
expressive student engagement. It is possible that more global strat- 
egies which would distinguish between teachers on the "how" of strategy 
use would produce more predictable effects, but this area, as well 
as the multiple variables associated with student engagement, remains 
as a Consideration for further research. 

Implications for In-Service Training 

Teaching is a very complex process. Given this fact, it seems 
reasonable that pre-professional training alone will not be sufficient 
to insure continued skill growth in practicing teachers. At the same 
time, teachers are very suspicious of in-service training that is not 
directly relevant to what they do in their every-day classroom situa- 
tions. 

This study demonstrated to teachers that it was possible to 
modify their instructional techniques by objectively monitoring stu-- 
dent responses in their classrooms. The method used was valuable in 
giving the teacher an objective view of her influence on her class. 
In the isolation of the classroom, teacher perspectives can become 
narrow and awareness of the effect of teacher behavior on students may 
be lost. 

The teachers in this study reported that they were not always 
sensitive to the cues of the class in determining possibilities and 
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limitations of different presentations. For example, Teacher 1 tended 
to judge the engagement level of her class by the amount of noise in 
the room. She admitted her desire to dominate, yet was able to see 
with feedback that when she relinquished some control the class re- 
sponded with enthusiasm. She finally stated that perhaps the perfect 
order she usually kept was not necessarily the ideal state in the 
classroom. The feedback and interaction with the observers was very 
important in her feeling comfortable with the new classroom climate. 

All the teachers enjoyed this project and the feedback it gave 
them. They seemed to be motivated and stimulated by ideas from the 
observer/ teacher interaction and reported that ideas were not imposed 
upon them. They chose ideas and implemented them in their own styles 
and in conjunction with their existing lesson plans. None reported 
feeling ''pushed** or complained about an extra work load. 

This model seems to be very useful for in-service training in 
that it deals with the individual teacher and her relationship to the 
class. The teacher is not made to feel that she should conform to a 
style that is not right for her, but her own strengths and weaknesses 
are the basis for suggested change. Making a staff of obserx'ers avail- 
able to teachers to record their behaviors and associated levels of 
student engagement might be a very useful way of improving instruction 
in classrooms. It would meet the scientific criterion of objective 
quantification and the practical criterion of immediate relevance. 
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TKACIIKR O BSKHVATTO M INSTItUMENT 

.Strategies will be marked with a v/for positive action and a — for constrnlnlnr. 
behavior. 

STRAT J ifijC .DEFiNrno: : 

St in^uLus y^r i a Oon jChan^ Tc^acher initiates chnnj;a in acti\ity anf'/nr 

subject mattC'.r. 
Examples : 

1. Teacher introduces a math lesson oy 
writing examples on the board, thoT. ha 
students work with f*,coboards, 

2. Teacher changes lesson from math (o 
reading, 

Surpriso /NovcJty Teacher does" something out of the ordiiiruy t 

arouse the curiosity and attention of iur 
students. 
Examples: 

1. Teacher uses poetry to illustrr.Lo 
history of contemporary problems, 

2. Teacher breaks routine by playing 
music to accompany the lesson. 

Kinesthetic Manipulation of Teacher involves students in activities or 
Materials 

* tasks requiring the manipulation of materials 
Examples: 

1. Students using gcoboards, blocks, 
c«i5?inaire rods. 

2. Student manipulates othnr student .i in 
front of the class to dcmonstiaio a 
math problem. 

l£:«l'L««_£5£.oJj:LM.al_^^^^ Teacher uses visual stimuli to facllif.n.o 

the lesson. 
Examples: 

1. Teacher uses charts, pictures, ovcrhr.u! 
projector. 

2. Teachrr writes on blackboard. 
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Teacher moves from one place of posiLion in 
classroom to another in order to facilitate 
the task or to interact uith student in t»isk 
related situation, 
Examp] cs: 

1. Teaclior cliechs on seat \;ork, 

2* Tc3c!ier walks around roi^>M from riu»lcnt 
to student vhen teaching a lesson. 

Teacher explains what the lesson will be r.bout, 
how it is related to what they liavo aln^ndy 
learned and witat will be expected of Lheia, 
Examples: 

1* ^ Teacher tells students about the gane 
they are going to play and why. 

2. Teacher discusses with students what is 
planned for today. 

Teacher offers clioiccs to students for self 

structuring of tasks. 

Examples; 

!• Wljat would you like to do today? 
2. Would you like to use the listening 
center or the math center? 

Teaclier describes or explains t.?nk, reds 
from a book, or answers a student's qncr.ii».a*. 
in informative terms. 
Ex«implcs : 

1. 2 + 2 " 4 and /i 2 = 2 

2* Abralutm Lincoln was a great president. 

Teacher pulls together and restates 5:01:1c lisped 
of the lesson or repeats student's answer. 
Exaniples: 

1. Jim says the answer Is four* 
2» Yesterday we discussed Tliomas Jefferson'^ 
attitude about land ownership and itp 
relevance to present day problems. 
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Imperative Teachef directs sLu<!onti; to do onl^ acaderoic 

tasks using commands or requests. 
Examples: 

1« Please open your books. 

2. Would you work that probleni on the boai'd? 

3. Will you sit down and finish your work? 
Personalizes Tadc Teacher relates task to students' or her own 

personal experience (s)« 
Examples: 

!• Teacher uses student homos to teach map 
reading or student names to teach 
alphabetizing, 

2. Teacher discusses previous IV show with 
class in relation to lesson. 

3. Today is John's birthday • 

4. Teacher speaks Spanish to clarify something 
to a primarily Spanish speaking student. 

Specific Qacg t ions Teacher askc a qvcstfcn roLTteH t:o r.c/!<*onic 

subject matter only for which there is one 
correct answer or a predetermined list of answers. 
Examples : 

1. How do you count to 10 in Spanish? 

2. Name the months of the year. 

3. Teacher poses problem — then asks student 
for answer using his name: John? 

Challenginf > Teacher asks open-ended, non-specific questions 

which involve thout»ht on the part of the student. 
Examples: 

1. How did we arrive at that answer? 

2. V/hat are ways that we can help save 
our enviroumiMit? 
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Feedback Teacher gives Infoiroation (verbally or nonverbally) 

about the accuracy of a student response. 
Examples : 

1. Head nod or shake in response. 

2. Goody OK, right, no, not quite, that*s wrong. 

3. Good, 3 + 4 a 7. 

Competency Testing Teacher gives test in order to assess student 

proficiency in specific subject matter. 
Kxamples: 

1. Math quiz. 

2. Spelling test. 

Competition Teacher divides class by individuals or groups 

for motivation for completing tasks and/or 
purposes of evaluation. 
Examples : 

1. Whoever Is finished first will leave 
for recess first. 

2. Spelling bee. 

3. Ilangman game. 

Personalizes Teacher focuses attention on an individual, child, 

creating a momentarily dyadic relationship uith 
one child. This can occur even across a ulassroon. 
Examples: 

1. Teacher moves from student to 'Student 
offering Individual help. 

2. Teacher uses student *s name. 

5iilLlp/^;.»>j2Jl Facial expression of teacher is one of ptoaeore 

and approval. 
Examples: 

1. Teacher r.milcs or laughs while interacting 
with Individual or c)ar.s. 
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Phys i cal Contnct 



Listens 



Feedback (SUC) 



Feedback (RIA) 



Teacher is involved In affncttynate ph>.-:ical 
Int-^raction or negative physical intciMctioa 
with student. 
Examples : 

1. Teacher puts arm around child, 

2. Teacher holds child's hand as sho 
helps the student to write letters, 

3. (Constraining) Teaehcr turns sludont 
around in desk sternly by arm. 

Teacher focuses attention on any student's 
verbal expression (eye contact is one measure) 
and indicates a real interest and concern. 

Communicates Expectations for Level of 
Success:- Teacher recognises ability to succeed 
either of an individual or the whole class. 
Examples: 

1. I just know Martha can do it, 

2. The whole class is going to do well 
on the tc:,r. 

Recognizes Individual Accomplishments: Teacher 
indicates that a student or class has performed 
well, has m.;de an unusual contribution, and/or 
has achieved more than the usual sta'.jdards of 
excellence. 
Examples : 

1. Look at how well Sandra is working. 

2. This is the first time you have done iti 
3# Jolni, you understand this. V^onld you 

help the others in your group? 

Any commands, requests, questions, or f.:cUcr.icnls 
that re J lite to non-aciidcmlc management or 
administration. 
Examples: 

3 . Close the door. 

2, The football team meets at 3, 

3. Please sit down and be quiet. 



APPENDIX B 

(1) Student Observation Instrument 

(2) Student Behavior Categories: Definitions and Examples 
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STuni:.N T OBSKRVATTo:; IN strumi:nt 

All ii. ^cnLou; vith tlic exception of "jjlobal assessment" will be marked 
vith a 

INnJCATOK DCFIN TTJON 

^£±SJi}M.VP.!l Each line of cvoi-y set will be numbered 

from 3 to 10, 

B Ulach 

M Mexican 

A Asian 

0 0Lhcri5 (American Indians, etc,) 

.^l^tSLE JIlV ''.vl^iiSJlI Student movement in relation to teacher 

directed task, 
Kxamp les: 

1. Student raising hand. 

2. Student writing. 

- GcuUciic muvci.u:ul xn relation to 

non-tasks. 
Examples: 

1. Student throwing things, fidgeting, 
hitting another child. 

2. Student walking around room withour. 
direction. 

Verb al Vav,\;v,'vvc:m Student recitation in response to 

r.pi'i Oiiriato ta^k. 
l->:a!.iples: 

1. Student answering teacher's 
questions, stating ideas, reciting- 

2. Gioup recitation, singing, 
practicinij phonic sounds. 
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VfM-hal D)Scnr.af:enont Student talking or sinj'.ing inappropriately. 

Examp] es : 

1. Stuflcnt talking to another sturlcnt 
about a non-task related subject. 

2. Student answering te.icher back 
v;hen not appropriate. 

Vifual Eng agcntent Student visually attending to the 

appropriate task. 
Examples : 

1. Student watching teacher while he 
is talking. 

2. • Student watching a board, filiu, 

or book if he is reading. 

Visual Dlre ngageTnent Student's eyes not focused on appropriate 

tasks. 
Example r« : 

1. Student looking at window, at: 
floor, or daydreaming. 

2. Student sleeping. 



A 1 or 2 intensity rating will be used to mark global assessment. 
A 2 indicates extreme intensity. 



G 1 ob a 1 Znz n g c ne n t 
Receptive 



Kx presri ve 



Student looking at appropriate object but 
showing no wotor or verbal behavior. 
Examples : 

1. Student listening to teacher. 

2. Student watching film or reading 
a book. 

Student looking at appropriate object while 
part j.cip.U ing in motor or verbal behavior. 
Examples : 

1* Student raising hatid to answer a q;te:5t io:i 
2. Student involved in wiiting. 
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Globnl Din cnfincemcnl 
Passive 



Disruptive 



Direction 
Non-Task 



Student not attending but not disturbing 

another student or class. 

Exainples: 

1. Student daydrecming or sleeping. 

2. Student drawing pictures wtien he 
should be reading. 

Student behaving in a manner disruptive to 
tlie learning process of one or more students. 



Student not attending to appropriate task. 
Examples: 

1. Student looking at book when he should 
• be attending to teacher or material. 

2. Student involved in bothering other 
students. 

When a student is involved in a non-task behavior, the non-task indicator and 
the indicator the student is supposed to be directed towards will be cht ckcd . 



Teacher 



Aide 



Material 



Student attending to teacher. 
Examples: 

1. Student and teacher working together 
on a problem. 

2. Student raising hand to attract 
teacher's at^-^ntion. 

Student interacting with teacher aide 

pr assistant. 

Examples: 

1. Adult helping student. 

2. Student from upper grade lielping 
one or more students. 

Student enga<^cd in an academically appropriate 
tack which involves use of materials. 
Examples: 

1. Student reading a book. 

2. Student working with a geoboard. 
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pp ors Student working LOi;othcr on an anpro- 

pri«^tc task without diroct adult 
supervision* 
Examples : 

1. Group of students working on a 
tnap study. 

2, Student listening to another 
student recite n poem. 

G rouping 

2 - student i.itcracting with one other person. 

Small - eight or less students. 
Large - nine or more students. 
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APPENDIX C 



Intcr-ohscrvor Agreement Tables 

Inter-observcr Reliability of Teacher Strategies and Student 
Behavior Categories Mean Adjusted Agreement over Ten Minute 
Calibration Period for All Rounds 

Inter-observer ReliJibility of Teacher Strategies Mean Adjusted 
Agreement over Ten Minute Calibration Period for all Observers 

Inter-observer Reliability of Student Behavior Categories Mean 
Adjusted Agreement for All Observers over Ten Minute Calibration 
Period 

Ii:ter-obsciver Reliability of Teacher Strategies Mean Adjusted 
Agreement over Ten Minute Calibration Period for All Observers 

Inter-observer Reliability of Student Behavior Categories Mean 
Adjusted Agreement over Ten Minute Calibration Period for All 
Observers 

Inter-observer Reliability of Global Student Behavior Categories 
Mean Adjusted Agreement for All Observers Over Ten Minute Cali- 
bration Period 
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TAnj-r: i 



Intur-ObservtM* lability of Teacher Strategies 
and Studt»nt Behavior Categories 
Mean Adjusted Agrcomcut over Ten Minute Calibration Period 
Across All Rounds for All Observers 



Teacher Stratecy 



Rd. I Rd. II 

X n=6 X n=6 



Mean 



Stimulus Variation 
and Change 


.96 


.99 


.98 


Si^rprises 


1.00 


1.00 


1.00 


Encourages Manipu- 
lation of Materials 


.6a 


.99 


.84 


Uses Visual 
Aides 


.75 


.80 


.78 


Moves 


.53 


.63 


.58 


Orients 


.98 


.98 


.98 


Choice 


1.00 


1,00 


1.00 


States /Explains 


.48 


.76 


.62 


Sumnarizes 


.75 


.87 


.81 


Corur.ands 


.75 


.78 


.77 


Personalizes 
Tar.!: 


.90 


.86 


.88 


Questions 


.63 


.77 


.70 


ChallenRes 


.92 


.89 


.91 


Gives 
Feedback 


.71 


.85 


.78 


TcstfJ 


1.00 


1.00 


1.00 


Fosters 
Co;npetition 


1.00 


1.00 


1.00 


P<:i"son.il ir.c s 


.76 


.86 


.81 


Smiles 


.90 


.83 


.8<) 


Touches 


.98 


.93 


.96 


List ens 


.58 


.66 


.62 


AuticipatCK 
S»!CCC5;s 


1.00 


.99 


1.00 


Rewards Indiv^Jual 
Arhi-vcincnL 


.92 


.95 


.94 


AdhUnistrat ion/ 
Mana»\ciiient 


.97 


.97 


.97 


Discipline 


.98 


.92 

^.89_ 


.95 
.87 


Ave rape /Round 


.«/»^ 





Student 
}5ehavlor 



Motor 

nngap^ed 



Motor 

Diseng aged 



Verbal 
En^^aged 



Verbal 
Di sengaged 



Visual 
Engaged 



Visual 
Disengaged 



Global 
R eceptive 



Global 
Kxpressive 



Global 
Passive 



Global 
Disruptive 



Direction 
Nc^n-tasl'. 



Direction 
Teacher 



Direction 
Aide 



Direction 
Materials 



Direction 
Peers 



Grouping 
Two 



Grouping 
Small 



Rd. I 

X n-6 



Grouping 
Lnrge 

Average /Round 



.80 



.67 



.86 



•a9 



.80 



.80 



.71 



.74 



.84 



.90 



.77 



.50 



1.00 



.74 



.95 



.96 



1.00 



.96 



.83 



Rd. TI 

X n -*6 



Mean 



,79 



.74 



.99 



.95 



.89 



.90 



.73 



.78 



.94 



.96 



.00 



.89 



1.00 



.74 



.91 



.91 



.96 



.88 



.88 



.80 



.71 



.93 
.92 



.85 



.85 



.72 



.76 



.89 



.93 



.«4 



.70 



1.00 



.74 



.93 



.94 



.98 



.92 



.86 
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Round I 
TAI5LK II 

Intcr-Obsci ver Reliability of Teacher Strategics 
Mean Adjusted Agrfreruont over 10 Minute Calibration Period 
for All Observers N=6 



Teacher Strategy 


Agree. 
X n-6 


No. of 
Differ. 


Av. Kreq. 
of Strat. 
Occur . 


Freq. oi Use 
Cols. & Obs. 


Range of Strategy 
Occurrence foi" 
Cals. 6. Obs. 


Stitaulus Variation 

and Chanj^c 


.96 


4 


C= .33 
0== .67 


C « 2 
0 » 4 


C^O.O - .03 
0-0.0 - .10 


Suri>- ir.fs 


1.00 


0 


C« 0.0 
0= 0.0 


C « 0 
0=0 


C^^O.O -0.0 
0^0.0 -0.0 


Kncourii^c:; !•!.'; nipu- 
lation of Materials 


.68 


29 


C-11.67 
0- 8.50 


C = 70 
0-51 


C- 0.0 -1.00 

0=^0.0 -1.0':^ 


Uses Visual 
Ai<^e*; 


.75 


23 


C« 9.17 
0^10.33 


C = 55 
0 « 62 


C^O.O - .63 
0= .03- .77 


Moves 


.53 


42 


C-23.50 
0=17.33 


C «141 
0 =104 


C= .60- .93 
0» .20- .83 


Orients 


.98 


2 


C= .50 
0= .50 


C « 3 
0 « 3 


C=0.0 - .07 
0=0.0 - .07 


Choice 


1.00 


0 


C= 0.0 
0- 0.0 


C = 0 
0=0 


C«0.0 -0.0 
0=0.0 -0.0 


States/Lxpli'ins 


./♦8 


47 


C=11.50 
0=10.33 


C = 69 
0 « 62 


C= .30- .50 
0- .07- .63 


Sumn-grizcs 


.75 


23 


C= 4.33 
0=» 4.17 


C = 26 
0 « 25 


C-0.0 - .43 
0-0.0 - .27 


Conmands 


.75 


23 


C«^ 8.17 
0= 7.17 


C = 49 
0 « 43 


C= .07- .60 
0= .03- .57 


Personalizes 
Task 


.90 


9 


1.00 
0« .50 


C « 6 
0 » 3 


C^O.O - .20 
0=0.0 - .10 


Question*: 


.63 


33 


C»11.67 
0= 9.17 


C « 70 
0 « 55 


C- .23- .87 
0= .13- .67 


Chall cnj^cs 


.92 


7 


C« 1.00 
0- .83 


C = 6 
0 « 5 


C=0.0 - .07 
0=0.0 - .13 


Gives 
Feedback 


.71 


26 


C« 5.17 
0= 3.83 


C « 31 
0 = 23 


C« .07- .27 
0=0.0 - .23 


Ter>ts 


1.00 


0 


C« 0.0 
0-= 0.0 


C « 0 
0-0 


C-0.0 -0.0 
0*0-0 -0.0 


Fosters 
Compctiticn 


1.00 


0 


C« 4.83 
0^ 4.83 


C = 29 
0 « 29 


C=0.0 - .97 
0=0.0 - .97 


Personnl iscs 


.76 


22 


C«20.00 
0=18.33 


C =120 
0 =110 


C= .20-1.00 
0« .17- .90 


Smi 1 er. 


.90 


9 


C= 1.50 
0= 1.67 


C = 9 
0 = 10 


C=0.0 - .13 
0-0.0 - .10 


Touches 


.98 


2 


C« 0.0 
0« . 33 


C » 0 
0=2 


C^O.O -0.0 
0=0.0 " .07 


List r- no. 


.58 


38 


C«17.83 
0=13.83 


C =107 
0 = 83 


C= .40-1.00 
0= .27- .83 


Antic- ipnt cs 
Succe.':r. 


1.00 


0 


C= .17 
0» .17 


C = 1 
0 « 1 


C«.O.Q - .03 
0^0.0 - .03 


Rewards Individual 
Achievement 


.92 


7 


C= 1.50 
0« 1.00 


C = 9 
0=6 


C=0.0 - .10 
0=0.0 - .07 


Administration/ 
Man:i};C';nfn t 


.97 


3 


C« 1.33 
0^ .83 


C = 
0=5 


C=0.0 - .17 
0«0.0 - .17 


^ Uij.ci l\ljnc. ^^^^ ^ 


.98 

RV. Jits J.-tmxx.^ 

.84 


2_ 

3 


C« .17 
0» . 50 


C « 1 

0 « 3^, _ 


C=0.0 - .03 
O'^O.O - .10 


Averaf.c/ltov.nd 
Totals 








3!)1 




C =812 
0 •689 
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TABLK III 
Round' I 

lutor-Obscrvor Reliability of Student Behavior Categories 
Mean Adjusted Agrccisent over Ten Minute Calibration Period 
for All Observers 
N«6 



* ">tor Knpr»^ed 


Agree. 
.80 


Ko. of 

iM^fer, 


Av . Frc(j. 
of Strat. 

O:?our . 


Freq. of Use 
CalK. S Oh<i. 


Rau^'c of Stratt'.,v 
Occurionce for 
Calf.. c< Oh-". 


16 


c« i;..3 

0- 11 . 83 


c =• y.i 

0 71 


C- - .73 
0- .07 " .70 


Motor Pisenpaged 


.67 


26 


C« 6.67 
0- 9.17 


C = 40 
0 - 55 


C= .07 - .50 
0- .13 - .57 


Verbal Eaguj^ed 


.86 


11 


0- 3.00 


L> if 

0 » 18 


0= .03 - .13 


Verbal Disengaged 


.89 


9 


C« .83 
0= 1.33 


C « 5 

0 =^ 8 


C-0.0 - .10 
0«0.0 - .10 


Vinual Engaged 


.80 


16 


C« 24.37 
0« 22.50 


C = 145 
0 « 135 


C« .73 - .90 

0-^ .'/o - .«:> 


Visual Disengaged 


.80 


16 


C= 2.33 
0= 4 . 00 


C - 14 
0 = 24 


C=0.0 - .13 
0- .07 .20 


Global Receptive 


.71 


23 


C= 11.33 
0- 10.17 


C = 68 
0 = 61 


C« .10 - .70 
0:= .07 - .60 


Global Expressive 


.74 


21 


C= 12.83 
0= 12.00 


C = 77 
0 « 72 


C« .10 - .73 
0= .10 - .70 




,8A 


13 


C- 1.50 
0= 3.00 


C = 9 
0 18 


C=0.0 - .10 
0= .03 ~ .17 


Global Disruptive 


.90 


8 


C« .83 
0= 1 . 50 


C « 5 
0 « 9 


C--0.0 - .07 
0- .03 - .13 


Direction Non-task 


.77 


18 


C« 2.33 
0« 4.33 


C = 14 
0 = 26 


C=0.0 - .13 
0- ,07 - .20 


Direction Teacher 


.50 


40 


C= 16.33 
0= 20.67 


C « 98 
0 = 124 


C= .27 - .80 
0-^ .53 - .80 


Direction Aide 


1.00 


0 


C« 0.0 
0^ 0.0 


C - 0 
0 « 0 


C-0.0 -0.0 
0=^0.0 -0.0 


Direction 
Material 5; 


.74 


21 


C- 33.00 
0- 13.3 7 


C « 78 
0 79 


C- .10 - .77 
0- J 7 - .77 


Direction Peers 


.95 


4 


C« .17 
0^ .83 


C - 1 
0 5 


C- 0.0 - .03 
0.-. 0.0 - .07 


Grouping T\xt 


.96 


3 


C- 2.00 
0= 1.50 


C « 32 

0 - 9 


C'- 0.0 - .39 

0-^ 0/0 - ,?y 


Grouping Spall 


1.00 


0 


C» 0.0 
0^ 0,0 


C = 0 

0 0 


C" 0.0 -0.0 
0^ 0.0 -0.0 


Grouping L:irge 


.96 


3 


C- 2'i.50 
0^^ 2S.O0 


C - 147 
0 « 150 


C« .60 - .90 
0^ .63 - .90 


Average/Round 


.83 








Totaln 




248 




C " 805 
0 « 864 
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TABLF fV 
Round It 

Inter-Obporvcr Reliability of Toa;:hor Strategics 
Mean Adjusted AgrocncnL over 10 Minute Calibration Period 








for All 


Observers 




Anroe. 


No. of 

Differ. 


Av. Frcq. 
of f.trat. 

Occ-wv . 


Freq. of Use 
Cals. & Obs. 


K«in^e of S t ra tt. j-jy 
Occurrence for 
Cals. & Obs. 


i^tituuluj; Variation 
r.:id C\tr\v.p^e 




] 


C- .67 
0= ,83 


C- A 
0-^ 5 


0.0 - ,07 
0= 0.0 - .07 




1 .00 


0 


C- 0.0 

0=^ 0.0 


C- 0 
0= 0 


C- 0.0 -0.0 
0= 0.0 -0.0 


Encourages M.^nlpu- 

P.n JLLI-:^ tcrials 
Unoi. Visual 
Aide?: 


.99 


1 


9.67 
0= 8.17 


C= 58 
0= 49 


C= 0.0 -1.00 
0=* 0.0 -1.00 


.80 


18 


C- J2.17 
0-^1 0, 3'j 

0 ^] ? ,00 


C- 73 
0=- 62 


C= ,03- ,67 
0= 0.0 - .60 


Moves 


.i>3 


33 


C- 89 
0^ 72 


0.0 - ,73 
0= 0,0 - ,70 


Orients 


.98 


2 




0= 6 
0=^ 6 


C= 0.0 - .07 
0== 0.0 - .07 


Choice 


1.00 


0 


C= 0.0 
0- 0.0 


C= 0 
0=^ 0 


C=' 0.0 -0.0 
0=^ 0.0 -0.0 


SLatcr./Kxplains 


.76 


22 


C-^U . 33 
0=10.33 


C= 68 
0= 62 


.10- ,80 
0=^ ,03- .80 


Sunmiarizcs 


.87 


12 


C- 6.17 
0--' 5.83 


C« 37 
0= 35 


C= ,07- .40 
0= ,03- ,37 


Coi.'in.'^.nds 


.78 


20 


6.67 
0-- A, 67 


C«= 40 
0-» 23 


C= ,07- .43 
0« 0.0 - .30 


Personalizes 
Task 


.86 


13 


C=17.67 

0 — -> . o J 


C^'i06 
0^ 95 


C« ,13- ,90 
0= ,1.3- ,80 


Questions 


.77 


21 


W3.1> 
Oj] 3 . 33 


0= 80 


C- ,2C- ,77 
0= .:jo- ,73 


ChallGnj»^cs 


.89 


10 


C= 2.33 
0- 2.33 


C= 14 
0= 14 


C» 0.0 - ,23 
0= 0.0 - .27 


Gives 
Feedback 


.85 


14 


C- 4.17 
0* 3-00 


C« 25 
0' 18 


C= ,03- ,20 
0= .03- ,20 


Tests 


1.00 


0 


C- 0.0 
(\- ft (\ 


C= 0 
t»« u 


0.0 -0.0 
0= 0.0 -0.0 


Fosters 
Compete ti on 


1.00 


0 


C= 0.0 
0= 0.0 


C*= 0 
0=. 0 


C« 0.0 -0.0 
0= 0.0 -0.0 


Personal ir.cs 


.86 


13 


C-23.17 
0=22.33 
C« 3,50 
0- 3.00 


C='139 
0'>134 


C= 57- 93 
0« .47-. 97 


Smiles 


.88 


11 


€=» 21 
0-^ 18 


C« 0.0 -,30 
0=* 0.0 -,20 


Toucbes 


.93 


6 


C« 1,67 
0- 2.33 


C-^ 10 
0= 14 


C« 0.0 -.30 
0« 0.0 -.43 


T.istcM.s 


.66 


31 


C-10.83 
0^17.J)0 


C«101 
0=102 


C=-. ,33-, 90 
0= .20-. 93 


Ant jcJ ,>ntcj* 


.99 


1 


0^^ 0.0 


C« 0 
0- 0 


C« 0,0 -0.0 
0= 0.0 -0.0 ' 


Reward '3 Tndiviiiual 
Achlc'Vomf lit: 


.95 


5 


2.00 
0- ] . 50 


C« 12 
0« 9 


C" ,03- ,1/ 
0« .03- .13 


Admin i Mral Ion/ 
Mjn.!AC':-o!it 


.97 


3 


C« .83 
0« . 67 


C- 5 
0= 4 


C» 0.0 - .13 
0" 0:0 - .07 


Disc 1 1,1 i;u' 


.92 


7 


C« 1.50 
Or .67 


C» 9 
0» 4 


C« 0.0 - ,13 
0«_0.0 - .07 


Avfn!?'o/Uftut)d 


.89 










Total.; 




2AA 




C"896 
0*811 
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TAniE. V 
Round II 



Intcr-Obscrver Reliability of Student Behavior Categories 
Mean Adjusted Agreement over Ten Minute Calibration Period 





for All 


Observers 




Studnnt lU-h tvlor 


Agree. 

X 


Ko. of 

Ulffor. 


Av, Frcq. 
oC Behav. 
Occur. 


Frcq. of Use 
CrJ.s. & Obs. 


Ha ace of lienavi 
Occurrence foi 
CaJt^. Oh-;. 


Motor Enp^aged 


.79 


17 


0^10.83 


C= 71 
0« 65 


0- .23- .43 


Motor Disenpagfd 


.74 


21 


C-16.83 
0^17.00 


C*=101 
0=102 


0= .30- .87 


Verbal Enga'^tc-d 


.99 


1 


C« 3.17 
0» 3.33 


C» 19 
0= 20 


0«0.0 - .23 


Verba] DisMiK^fitid 


.95 


4 


C« 1.17 
0= 1.17 


C« 7 
0= 7 


0=0.0 - .07 


Visual Kngaj^ed 


.89 


9 


C=24 . 83 
0^-24.67 


C-149 
0=148 


0= .67- .93 


Visual Disong.^ged 


.90 


8 


C« 2.33 
0-- 2.17 


C= 14 
0» 13 


fisA 0 - 17 

C-0.0 - .20 


Global Receptive 


.73 


22 


C*11.67 
0«13.17 


C= 70 
0= 79 


0« .37- .50 


Global Expressive 


.78 


18 


0-11.67 


0- 70 


V"" . J J** . Uv 

0^ .2;- .47 


Clot al Passive 


.94 


3 


r« A 7 
u« 1.17 


0» 7 


/•-A n 07 
0-0.0 - .13 


Global Disruptive 


.96 


3 


0« 1.00 


0» 6 


0:^0.0 - .07 


Direction Non-Lask 


.90 


8 


Pr- 7 17 

0=: 2.17 


C= 13 
0=* 13 


C=0.0 - .17 
0=^0.0 - .20 


Direction Teacher 


.89 


9 


C«22.33 
0=«22.50 


3-135 


0» .47-- .90 


Direction Aide 


1.00 


0 


0« 0.0 


C« 0 
0« 0 


0= 0.0 -0.0 
0^0.0 -0.0 


Direction 
Materials 


.74 


21 


0*11.17 
0« 7.83 


C» 67 
0« 47 


C«0.0 - .80 
0-0.0 - .67 


Direction Peers 


.91 


7 


C" 1.00 
0- 1.83 


C= 6 
0= 11 


C-0.0 - .13 
O- O.O " .17 


Croup inp. Two 


.91 


7 


1.33 
0^ .b3 


C« 8 
0^ 5 


C-0.0 - .10 
0-0.0'- .J7 


Grouping Small 


.96 


3 


C« .50 
0«^ 0.0 


C« 3 
0« 0 


C«0.0 " .07 
O'-O.O -0.0 


Grouping Lar^c 


.88 


10 


C»25.17 
0=26.17 


C-151 
0==157 


C« .80- .87 
0=« .80-1.00 


Average/RouiV.! 


.88 










Totals 




173 




C«-*905 
0"885 
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TABLE VI 

Inter-Observer Reliability of Global Student Behavior Categories 
Mean Adjusted Agre&nent 
for All Observers over Tea Minute Calibration Period 











Global Behavior 


Round I 


Round II 


Kean 


kcccptlvc 


.71 


.73 


.72 


Expressive 


.74 


.78 


.76 


Passive 


.84 


.94 


.89 


Disruptive 


.90 


.96 


.93 


Averagc/Kound 


.80 


.85 


.83 











